This letter points out that while a class of conditions presented in Matsuoka K. [1] are truly sufficient for absolute stability of neural networks, the proof of the sufficiency given in [1] is not sound. As a remark, a mathematically rigorous proof of the sufficiency of the class of conditions for absolute stability of neural networks is provided.

A main result in this paper is that for a Hopfield-type neural circuit with a symmetric connection matrix T, the negative semidenfiniteness of T is a necessary and sufficient condition for absolute exponential stability. While this result extends one of absolute stability in Forti, et al. [1], its proof given in this paper is simpler, which is completed by an approach different from one used in Forti et al. [1]. The most significant consequence is that the class of neural networks with negative semidefinite matrices T is the largest class of symmetric networks that can be employed for embedding and solving optimization problem with global exponential rate of convergence to the optimal solution and without the risk of spurious responses.

This paper describes a new and realisable speech enhancement structure which simulates the cocktail party effect with a modified iterative Wiener filter and a multi-layer perceptron neural network. The key idea is to use the neural network as a speaker recognition system to govern the iterative Wiener filter. The neural network is a modified perceptron with a hidden layer using feature date extracted from LPC cepstral analysis. The proposed technique has been successfully used for speech enhancement when the interference is competing speech or broad band noise.

In this paper, a simple method to investigate the dynamics of continuous-time neural networks based on the force (kinetic vector) derived from the equation of motion for neural networks instead of the energy function of the system has been described. The number of equilibrium points and limit cycles of one-dimensional neural networks with the asymmetric cyclic connection matrix has been investigated experimently by this method. Some types of equilibrium points and limit cycles have been theoretically analyzed. The relations between the properties of limit cycles and the number of connections also have been discussed.

Neural network hardware using time-shared bus and integer representation architecture has already been fabricated and reported from the design viewpoint. However, nothing related to performance evaluation of hardware has yet been presented. Computation-speed, scalability and learning accuracy of hardware are evaluated theoretically and experimentally using a Back Propagation (BP) algorithm. In addition, a mirror-weight assignment technique is proposed for high-speed computation in the BP. NETTalk, an English-pronunciation-reasoning task, has been chosen as the target application for the BP. In the experiment, recently-developed neuro-hardware based on the above architecture and its parallel programming language are used. An outline of the language is described along with BP programming. Mirror-weight assignment allows maximum speed at 55.0 MCUPS (Million Connections Updated Per Second) using 256 neurons in the hidden-layer (numbers of neurons in input-and output-layers are fixed at 203 and 26 respectively in NETTalk). In addition, if scalability is defined as a function of the number of neurons in the hidden-layer, the machine retains high scalability at 0.5 if such a maximum speed needs to be used. No degradation in learning accuracy occurs when experimental results computed using the neuro-hardware are compared with those obtained by floating-point representation architecture (workstation). The experiment indicates that the present integer representational design of the neuro-hardware is sufficient for NETTalk. Performance has been evaluated theoretically. For evaluation purposes, it is assumed that most of the total execution-time is taken up by bus cycles. On the basis of this assumption, an analytical model of computation-speed and scalability is proposed. Analytical predictions agreed well with experimental results.

In earlier works we proposed the Exclusive Learning neural NET work (ELNET), which can be utilized to construct large scale recognition system for Chinese characters. However, this did not resolve the problem of how to use training samples effectively to generate more suitable recognition boundaries. In this paper, we propose ELNET- wherein an attempt has been made to deal with this problem. In comparison with ELNET, selection method of training samples is improved. And the number of module size are variable according to the number of training samples for each module. In recognition experiment for ETL9B (3036 categories) using ELNET-, we obtained a recognition rate of 95.84% as maximum recognition rate. This is the first time that such a high recognition rate has been obtained by neural networks.

A new neural network for locating a source by integrating data from a number of sensors is considered. The network gives a solution for inverse problems using a back-propagation algorithm with the architecture to get the solution in the inter-layer weights in a coded form Three different physical quantities are applied to the network, since the scheme has three independent ports; an input port, a tutorial port and an answer port. Our architecture is useful to estimate z" in the problem whose structure is y=f(x,z) where y is the observed data, x is the sensor position and z is the source location. The network integrates the information obtained from a number of sensors and estimates the location of the source. We apply the network to two problems of location estimation: the localization of the active nerves from their evoked potential waveforms and the localization of objects from their echoes using an active sonar system.

Devanagari is the most widely used script in India. Here, a method is introduced for recognizing Devanagari characters using Neural network. The proposed method reduces the number of output unit necessary for a conventional neural network where the classification is based on a winner take all basis. An automatic coding procedure for representing the output layer of the network and a different method for the final classification is also proposed. Along with the automatic coding procedure, a heuristic method for representing the output units by exploiting the structural information of Devanagari character is also demonstrated. Besides, it has been shown by random representation of the output layer that the representation effects the generalization/performance of the network. The proposed automatic representation gave the recognition rate of 98.09% for 44 categories.

The mean field theory has been recognized as offering an efficient computational framework in solving discrete optimization problems by neural networks. This paper gives a formulation based on the information geometry to the mean field theory, and makes clear from the information-theoretic point of view the meaning of the mean field theory as a method of approximating a given probability distribution. The geometrical interpretation of the phase transition observed in the mean field annealing is shown on the basis of this formulation. The discussion of the standard mean field theory is extended to introduce a more general computational framework, which we call the generalized mean field theory.

In this paper, we present some contributions to improve a previous work's approach presented for the segmentation of magnetic resonance images of the human brain, based on the unsupervised Hopfield neural network. We formulate the segmentation problem as a minimization of an energy function constructed with two terms, the cost-term as a sum of errors' squares, and the second term is a temporary noise added to the cost-term as an excitation to the network to escape from certain local minimums and be more close to the global minimum. Also, to ensure the convergence of the network and its utility in clinic with useful results, the minimization is achieved with a step function permitting the network to reach its stability corresponding to a local minimum close to the global minimum in a prespecified period of time. We present here our approach segmentations results of a patient data diagnosed with a metastatic tumor in the brain, and we compare them to those obtained based on, previous works using Hopfield neural networks, Boltzmann machine and the conventional ISODATA clustering technique.

A calibrating analog-to digital (A/D) converter employing a T-Model neural network is described. The T-Model neural-based A/D converter architecure is presented with particular emphasis on the elimination of local minimum of the Hopfield neural network. Furthermore, a teacher forcing algorithm is presented and used to synthesize the A/D converter and correct errors of the converter due to offset and device mismatch. An experimental A/D converter using standard 5-µm CMOS discrete IC circuits demonstrates high-performance analog-to-digital conversion and calibrating.

In this study, we discuss a discrete-time cellular neural network (DTCNN) and its applications including convergence property and stability. Two theorems about the convergence condition of nonreciprocal non-uniform DTCNNs are described, which cover those of reciprocal one as a special case. Thus, it can be applied to wide classes of image processings, such as associative memories, multiple visual patterns recognition and others. Our DTCNN realized by the software simulation can largely reduce the computational time compared to the continuous-time CNN.

A direct gradient descent learning algorithm of energy function in Hopfield neural networks is proposed. The gradient descent learning is not performed on usual error functions, but the Hopfield energy functions directly. We demonstrate the algorithm by testing it on an analog-to-digital conversion and an associative memory problems.

One of the major open issues in neural network research includes a Network Designing Problem (NDP): find a polynomial-time procedure that produces minimal structures (the minimum intermediate size, thresholds and synapse weights) of multilayer threshold feed-forward networks so that they can yield outputs consistent with given sample sets of input-output data. The NDP includes as a sub-problem a Network Training Problem (NTP) where the intermediate size is given. The NTP has been studied mainly by use of iterative algorithms of network training. This paper, making use of both rate distortion theory in information theory and linear algebra, solves the NDP mathematically rigorously. On the basis of this mathematical solution, it furthermore develops a mathematical solution Procedure to the NDP that computes the minimal structure straightforwardly from the sample set. The Procedure precisely attains the minimum intermediate size, although its computational time complexity can be of non-polynomial order at worst cases. The paper also refers to a polynomial-time shortcut to the Procedure for practical use that can reach an approximately minimum intermediate size with its error measurable. The shortcut, when the intermediate size is pre-specified, reduces to a promising alternative as well to current network training algorithms to the NTP.

Cellular Neural Networks (CNNs) are nonlinear dynamic array processors with mainly local interconnections. In most of the applications, the local interconnection pattern, called cloning template, is translation invariant. In this paper, an optimal ring-coding method for rotation invariant description of given set of objects, is introduced. The design methodology of the templates based on the ring-codes and the synthesis of CNN analogic algorithms to detect standing and moving objects in a rotationally invariant way, discussed in detail. It is shown that the algorithms can be implemented using the CNN Universal Machine, the recently invented analogic visual microprocessor. The estimated time performance and the parallel detecting capability is emphasized, the limitations are also thoroughly investigated.

We present a framework for the unsupervised segmentation of time series. It applies to non-stationary signals originating from different dynamical systems which alternate in time, a phenomenon which appears in many natural systems. In our approach, predictors compete for data points of a given time series. We combine competition and evolutionary inertia to a learning rule. Under this learning rule the system evolves such that the predictors, which finally survive, unambiguously identify the underlying processes. The segmentation achieved by this method is very precise and transients are included, a fact, which makes our approach promising for future applications.

The number of solutions of a nonlinear equation x = sgn(Wx) is discussed. The equation is derived for the determination of equilibrium points of a kind of Hopfield neural networks. We impose some conditions on W. The conditions correspond to the case where a Hopfield neural network has n neurons arranged on a ring, each neuron has connections only from k preceding neurons and the magnitude of k connections decrease as the distance between two neurons increases. We show that the maximum number of solutions for the above case is extremely few and is independent of the number of neurons, n, if k is less than or equal to 4. We also show that the number of solutions generally increases exponentially with n by considering the case where k = n-1.

We propose an algorithm for image decomposition based on Hadamard functions, realized by answer-in-weights neural network, which has simple architecture and is explored with steepest decent method. This scheme saves memory consumption and it converges fast. Simulations with least mean square (LMS) and absolute mean (AM) errors on a 128128 image converge within 30 training epochs.

The dynamics of an artificial neural network derived from a biological system, and its two applications to engineering problems are examined. The model has a multi-layer structure simulating the primary and secondary components in the olfactory system. The basic element in each layer is an oscillator which simulates the interactions between excitatory and inhibitory local neuron populations. Chaotic dynamics emerges from interactions within and between the layers, which are connected to each other by feedforward and feedback lines with distributed delays. A set of electroencephalogram (EEG) obtained from mammalian olfactory system yields aperiodic oscillation with 1/f characteristics in its FFT power spectrum. The EEG also reveals abrupt state transitions between a basal and an activated state. The activated state with each inhalation consists of a burst of oscillation at a common time-varying instantaneous frequency that is spatially amplitude-modulated (AM). The spatial pattern of the activated state seems to represent the class of the input ot the system, which simulates the input from sensory receptors. The KIII model of the olfactory system yields sustained aperiodic oscillation with "1/f" spectrum by adjustment of its parameters. Input in the form of a spatially distributed step funciton induces a state transition to an activated state. This property gives the model its utility in pattern classification. Four different methods (SD, RMS, PCA and FFT) were applied to extract AM patterns of the common output wave forms of the model. The pattern classification capability of the model was evaluated, and synchronization of the output wave form was shown to be crucial in PCA and FFT methods. This synchronization has also been suggested to have an important role in biological systems related to the information extraction by spatiotemporal integration of the output of a transmitting area of cortex by a receiving area.

The operation scheduling is an important subtask in the automatic synthesis of digital systems. Many greedy heuristics have been proposed for the operation scheduling, but they cannot find the globally best schedule. In this paper we present an algorithm to construct near optimal schedules. The algorithm combines characteristics of simulated annealing and neural networks. The neural network used in our scheduling algorithm is similar to that proposed by Hellstrom et al. However, while the problems of Refs. [11] and [12] have a single type of constraint, the problem considered in this paper has three types of constraints. As the result, the energy function of the proposed neural network is given by the weighted sum of three energy functions. To minimize the weighted sum of two or more energy functions, conventional methods try to find a good set of weights using a try and error method. Our algorithm takes a different approach than these methods. Results of the experiments show that the proposed algorithm can be used as an alternative heuristic for solving the operation scheduling problem. In addition, the proposed algorithm can exploit the inherent parallelism of the neural network.